Papers with latent variable models
Variational Inference and Deep Generative Models (P18-5)
Copied to clipboard
| Challenge: | Unsupervised and semi-supervised learning has been addressed scarcely in NLP . this tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
| Approach: | This tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
| Outcome: | This tutorial provides an introduction to variational inference followed by an example-driven discussion of how to use variational methods for training DGMs. |
Unsupervised Bilingual Lexicon Induction via Latent Variable Models (D18-1)
Copied to clipboard
| Challenge: | Existing methods for generating bilingual lexicons use parallel corpora or bilingual dictionaries. |
| Approach: | They propose a model which builds cross-lingual dictionaries using latent variable models and adversarial training with no parallel corpora. |
| Outcome: | The proposed model outperforms state-of-the-art models on several language pairs and reaches competitive performance. |
ThinkSum: Probabilistic reasoning over sets using large language models (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a substantial capacity for high-level analogical reasoning, but they fail in scenarios that require reasoning over multiple objects or facts and making sequences of logical deductions. |
| Approach: | They propose a two-stage probabilistic inference paradigm, ThinkSum, which reasons over sets of objects or facts in a structured manner. |
| Outcome: | The proposed paradigm improves on the BIG-bench suite of evaluation tasks. |
Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement (D18-1)
Copied to clipboard
| Challenge: | Despite its success, neural autoregressive modeling has its weakness in decoding, i.e., finding the most likely sequence. |
| Approach: | They propose a conditional non-autoregressive neural sequence model based on iterative refinement based upon latent variable models and conditional denoising autoencoders. |
| Outcome: | The proposed model significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart. |
Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue models lack prior and posterior knowledge selection . prior selection module may not learn to select knowledge properly because of lack of posterior information . |
| Approach: | They propose a knowledge distillation-based training strategy to remove the exposure bias of knowledge selection. |
| Outcome: | The proposed model improves on two knowledge-grounded dialogue datasets. |
FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow (D19-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models are autoregressive, meaning they factor the joint probability of the output sequence into the product of probabilities over the next to-ken. |
| Approach: | They propose a non-autoregressive sequence generation model using latent variables . they use generative flow to model complex distributions using neural networks . |
| Outcome: | The proposed model performs comparable to state-of-the-art models and has constant decoding time w.r.t the sequence length. |
Simple and Effective Noisy Channel Modeling for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Previous work on noisy channel modeling relied on latent variable models that incrementally process the source and target sentence. |
| Approach: | They propose to use a standard sequence to sequence model which utilizes the entire source and target sentences to estimate posterior probability of a target sequence y given a source sequence x. |
| Outcome: | The proposed model outperforms direct models on German-English translations by up to 3.2 BLEU on four language pairs. |